A Process to Calculate the Uncertainty of Software Metrics-based Models Using Bayesian Networks

ion sheets Step 2 Figure 1. Process overview. application domain and development process) from the domain experts. For instance, semi-structured interviews with project leaders might be performed. With the context defined, the next step is to apply the GQM process with the goal of capturing the experience of the viewpoints and selecting the software metrics. For this purpose, abstraction sheets must be used as knowledge acquisition instrument during interviews [3]. A set of goals is defined as G = {g1, . . . , g|G|}, where gi represents a project goal. For each goal, an abstraction sheet, which is composed of four quadrants, must be used. A set of questions, Q, and metrics, M , must be extracted from the first quadrant, which corresponds to the quality focus. qi ∈ Q and mi ∈M represent, respectively, a question and a metric related to a project goal. The second quadrant corresponds to the variation factors, which are factors that impact the quality focus, considering the defined goal. More questions q and metrics m can be extracted from this quadrant. The third quadrant corresponds to the possible values of the extracted metrics from the first quadrant. These values are important because they demonstrate the usefulness of the measurement process. By analyzing them, we can detect discrepancies between expectations and realities. Finally, the last quadrant of the abstraction sheet corresponds to the impact of the variation factors on the extracted metrics from the first quadrant. The description of this impact serves as motivation for the inclusion of the variation factor in the abstraction sheet. If the project leader does not know how to inform the impact of a variation factor, this factor should be excluded from the sheet. Given that each question must be associated with at least one goal, a set of relationships between goals and questions, T , should be created. t(g, q) means the goal g and question q are related. An hierarchical structure example of the GQM model is illustrated in Figure 2. For the example shown in Figure 2, the given goal is to “Analyze the software product with respect to its quality for the purpose of characterization from the developer’s point of view”. For quality focus, the question “How many unwanted behaviors does the product have?” was defined. For the variation factor, the question “What is the quality of the test?” was defined. An example of an abstraction sheet is illustrated in Figure 3. By analyzing first quadrant of the abstraction sheet presented in Figure 3, it is possible to identify metrics such as number of detected failures, proportion of critical/uncritical failures and number of detected faults. Given that the goal, questions and metrics for the quality of focus are defined, the Bayesian network can be built. For our approach, all node should be modelled as ranked. If a metric is collected using a numerical scale, thresholds must be defined to convert it into an ordinal scale. For this purpose, statistics-based approach [17] can be used or data must be collected from domain experts. After the construction of the directed acyclic graph, the NPT of the goal node must be defined. Assuming that the goal was modeled as a ranked node, we can create a truth table to collect data from a domain expert and define the NPT. For the given example, given that there is a one-to-one relationship between g and q, the truth table is not necessary and the NPT must be calibrated as an identity matrix, in which the diagonal elements are 1 and the remaining are 0. 3.2 Software Metrics Validation The validation of the metrics ensures that they are representative of the measured attributes. In the literature, there are many researches on the validation of software metrics [16, 12, 14]. Meneely et al. [14] performed a systematic review about validation criteria for software metrics and identified 47 criteria. In Table 1, we present ten criteria identified by Meneely et al. [14]. A priori validity Monotonicity Actionability Metric Reliability Appropriate Continuity Non-collinearity Appropriate Granularity Non-exploitability Association Non-uniformity Table 1. List of 10 validation criteria found in the review [14]. Defining the purpose of using a metric is a critical step to validate it. In addition, when the project leader makes a decision, he can specify properties of the metrics that are most appropriate to use. According to Meneely et al. [14] this is called advantage. For instance, be able to show that a metric is a significant representation and that it can be applied to a development process are considered advantages. Analyze the software product with respect to its quality for the purpose of characterization from the developer's